A stochastic Ramsey theorem

A stochastic extension of Ramsey’s theorem is established. Any Markov chain generates a filtration relative to which one may define a notion of stopping time. A stochastic colouring is any fc-valued (k < oo) colour func­ tion defined on all pairs consisting of a bounded stopping time and a finite partial history of the chain truncated before this stopping time. For any bounded stopping time 6 and any infinite history u> of the Markov chain, let oj\6 denote the finite partial history up to time 0(u). It is first proved for k = 2 that for every e > 0 there is an increasing sequence 9\ < 9<i < ... of bounded stopping times having the property that, with probability greater than 1—e, the history uj is such that the values assigned to all pairs (u\Qi, 6 j ), with i < j , axe the same. Just as with the classical Ramsey theorem, an analogous finitary stochastic Ramsey theorem is obtained. Furthermore, with appropriate finiteness assumptions, the time one must wait for the last stopping time (in the finitary case) is uniformly bounded, independently of the probability transitions. The results are generalised to any finite number k of colours. A stochastic extension is derived for hypergraphs, but with rather weaker conclusions. The stochastic Ramsey theorem can be applied to the expected utility of a Markov chain to conclude that on some infinite increasing sequence of bounded stopping times the expected utility remains the same to within e (also in probability).